docs: update incremental benchmarks (3.9.6)#1032
Conversation
Greptile SummaryThis automated PR appends 3.9.6 incremental benchmark results (native and WASM build, no-op rebuild, 1-file rebuild, and import-resolution metrics) to the summary table and latest-results section. It also drops the Confidence Score: 5/5Safe to merge — documentation-only change with internally consistent data No P0 or P1 issues found. The JSON payload, rendered tables, and latest-results section are mutually consistent. Previously identified concerns (missing 3.9.5 explanation, unannotated WASM regressions) are already tracked as follow-up issues and are not re-flagged here. No files require special attention Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
WF["GitHub Actions\nWorkflow #786"] --> |"benchmark run"| RAW["Raw timing data\n(native + WASM engines)"]
RAW --> GEN["update-incremental-report.ts\n(generator script)"]
GEN --> JSON["Embedded JSON block\n(INCREMENTAL_BENCHMARK_DATA)"]
GEN --> TABLE["Summary table\n(top-level version history)"]
GEN --> LATEST["Latest results section\n(per-metric breakdown + phases)"]
JSON --> MD["INCREMENTAL-BENCHMARKS.md"]
TABLE --> MD
LATEST --> MD
MD --> |"PR #1032"| MAIN["main branch"]
Reviews (3): Last reviewed commit: "Merge branch 'main' into benchmark/incre..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
|
||
| ### Latest results | ||
|
|
||
| **Version:** 3.9.5 | **Files:** 742 | **Date:** 2026-04-23 | ||
| **Version:** 3.9.6 | **Files:** 744 | **Date:** 2026-04-30 |
There was a problem hiding this comment.
The <!-- NOTES_START --> block that explained why 3.9.5 is absent from the top-level summary table was deleted, but 3.9.5 still has no row in that table (its JSON entry retains "wasm": null, "native": null). Anyone reading the table after this PR will see the history jump from 3.9.4 straight to 3.9.6 with no explanation. The note could either be kept as-is or condensed to a brief parenthetical on the 3.9.5 JSON entry.
There was a problem hiding this comment.
Tracked as a generator bug rather than a fix-in-place on this auto-generated file: #1042. The note that PR #1008 added was a manual edit, but scripts/update-incremental-report.ts regenerates the file from scratch on every release and has no awareness of <!-- NOTES_START --> sentinels — so any fix here would just be wiped out by the next benchmark run. The follow-up issue covers preserving the notes block on regeneration.
| | Version | Engine | Files | Full Build | No-op | 1-File | Resolve (native) | Resolve (JS) | | ||
| |---------|--------|------:|-----------:|------:|-------:|------------------:|-------------:| | ||
| | 3.9.6 | native | 744 | 3.0s | 10ms | 54ms | 7ms ~ | 11ms ↑7% | | ||
| | 3.9.6 | wasm | 744 | 14.0s | 131ms | 62ms | 7ms ~ | 11ms ↑7% | |
There was a problem hiding this comment.
The WASM engine shows significant regressions vs 3.9.4 (the last version with build data): full build went from 7.6s → 14.0s (~84% slower) and no-op rebuild from 19ms → 131ms (~589% slower). Because 3.9.5 had null build metrics, the generation script produced no ↑/↓ annotations for these columns in the 3.9.6 WASM row, making the regressions invisible to casual readers of the table. It may be worth checking whether the generator should fall back to comparing against the nearest version that has build data when the immediately prior version has nulls.
There was a problem hiding this comment.
Tracked as a generator improvement: #1043. The blank trend cells are caused by findPrevRelease returning the immediately prior release (3.9.5) which has null build/rebuild metrics, so trend() short-circuits to an empty string. The follow-up issue covers walking further back per-metric to find the nearest release with a non-null value. Worth flagging that the wasm numbers themselves may be partly environmental — the same investigation noted in #1042 about the 3.9.5 SIGKILLs is the right place to assess whether 3.9.6 wasm is a real regression or runner noise.
…-v3.9.6-20260430-065206
1 participant
Automated incremental benchmark update for 3.9.6 from workflow run #786.